Papers by Elad Ben Zaken
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models (2022.acl-short)
Copied to clipboard
| Challenge: | Large pre-trained models are expensive to train and deploy . large data sets make finetuning expensive to deploy - a new paradigm . |
| Approach: | They propose a sparse-finetuning method where only bias-terms are modified . they show that applying BitFit on pre-trained BERT models is competitive . |
| Outcome: | The proposed method is competitive with sparse-finetuning methods on small-to-medium training data. |